Unsupervised Bayesian Parameter Estimation for Dependency Parsing

نویسندگان

  • Shay B. Cohen
  • Kevin Gimpel
  • Noah A. Smith
چکیده

We explore a new Bayesian model for probabilistic grammars, a family of distributions over discrete structures that includes hidden Markov models and probabilitsic context-free grammars. Our model extends the correlated topic model framework to probabilistic grammars, exploiting the logistic normal prior as a prior over the grammar parameters. We derive a variational EM algorithm for that model, and then experiment with the task of unsupervised grammar induction for natural language dependency pasring, and show that our model achieves superior results over previous models that differ mostly in the choice of a prior different than the logistic-normal prior. For a full version of this paper, see [5].

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

An Unsupervised Parameter Estimation Algorithm for a Generative Dependency N-gram Language Model

We design a language model based on a generative dependency structure for sentences. The parameter of the model is the probability of a dependency N-gram, which is composed of lexical words with four kinds of extra tags used to model the dependency relation and valence. We further propose an unsupervised expectationmaximization algorithm for parameter estimation, in which all possible dependenc...

متن کامل

A Generative Dependency N-gram Language Model: Unsupervised Parameter Estimation and Application

We design a language model based on a generative dependency structure for sentences. The parameter of the model is the probability of a dependency N-gram, which is composed of lexical words with four types of extra tag used to model the dependency relation and valence. We further propose an unsupervised expectation-maximization algorithm for parameter estimation, in which all possible dependenc...

متن کامل

Efficient Inference for Unsupervised Semantic Parsing

Unsupervised semantic parsing algorithms based on Bayesian nonparametric grammars offer a promising way to bootstrap semantic analyses for unsupervised relation/information extraction (RE/IE). Yet the form in which they currently exist makes it difficult to apply them to larger data sets. In this paper, we make progress toward scaling unsupervised semantic parsing by introducing a local, collap...

متن کامل

Bayesian Learning of Non-compositional Phrases with Synchronous Parsing

We combine the strengths of Bayesian modeling and synchronous grammar in unsupervised learning of basic translation phrase pairs. The structured space of a synchronous grammar is a natural fit for phrase pair probability estimation, though the search space can be prohibitively large. Therefore we explore efficient algorithms for pruning this space that lead to empirically effective results. Inc...

متن کامل

Bayesian Learning of Non-Compositional Phrases with Synchronous Parsing

We combine the strengths of Bayesian modeling and synchronous grammar in unsupervised learning of basic translation phrase pairs. The structured space of a synchronous grammar is a natural fit for phrase pair probability estimation, though the search space can be prohibitively large. Therefore we explore efficient algorithms for pruning this space that lead to empirically effective results. Inc...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2008